<html>
<head>
  <title>异步 I/O 框架</title>
  <basefont face="微软雅黑" size="2" />
  <meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
  <meta name="exporter-version" content="Evernote Windows/307027 (zh-CN, DDL); Windows/6.1.0 (Win32);"/>
  <style>
    body, td {
      font-family: 微软雅黑;
      font-size: 10pt;
    }
  </style>
</head>
<body>
<a name="627"/>
<h1>异步 I/O 框架</h1>

<div>
<span><div><div><a href="http://python.jobbole.com/87310/" style="font-weight: bold; font-size: 14pt; color: rgb(173, 0, 0);">http://python.jobbole.com/87310/</a></div><div><span style="font-size: 14pt; color: rgb(173, 0, 0); font-weight: bold;"><br/></span></div><div><span style="font-size: 14pt; color: rgb(173, 0, 0); font-weight: bold;">CPU高速执行能力和IO设备的龟速严重不匹配</span></div><div><span style="font-size: 12pt; color: rgb(51, 51, 51); font-family: Monaco;">多线程和多进程只是解决这一问题的一种方法。</span></div><div><br/></div><div><span style="font-size: 12pt; font-family: Monaco;">另一种解决IO问题的方法是</span><span style="font-size: 12pt; color: rgb(173, 0, 0); font-family: Monaco; font-weight: bold;">异步IO</span><span style="font-size: 12pt; font-family: Monaco;">。当代码需要执行一个耗时的IO操作时，它只发出IO指令，并不等待IO结果，然后就去执行其他代码了。一段时间后，当IO返回结果时，再通知CPU进行处理。</span></div><div><br/></div><div>可以想象如果按普通顺序写出的代码实际上是没法完成异步IO的：</div><div style="box-sizing: border-box; padding: 8px; font-size: 12px; border-top-left-radius: 4px; border-top-right-radius: 4px; border-bottom-right-radius: 4px; border-bottom-left-radius: 4px; background-color: rgb(251, 250, 248); border: 1px solid rgba(0, 0, 0, 0.14902);"><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 12px;">do_some_code()</span></div><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 12px;">f = open('/path/to/file', 'r')</span></div><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 12px;">r = f.read() # &lt;== 线程停在此处等待IO操作结果</span></div><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 12px;"># IO操作完成后线程才能继续执行:</span></div><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 12px;">do_some_code(r)</span></div></div><div><br/></div><div>异步IO模型需要一个消息循环，在消息循环中，主线程不断地重复“读取消息-处理消息”这一过程：</div><div style="box-sizing: border-box; padding: 8px; font-size: 12px; border-top-left-radius: 4px; border-top-right-radius: 4px; border-bottom-right-radius: 4px; border-bottom-left-radius: 4px; background-color: rgb(251, 250, 248); border: 1px solid rgba(0, 0, 0, 0.14902);"><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 12px;">loop = get_event_loop()</span></div><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 12px;">while True:</span></div><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 12px;">    event = loop.get_event()</span></div><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 12px;">    process_event(event)</span></div></div><div><br/></div><div><br/></div><div><font style="font-size: 18pt;"><span style="color: rgb(173, 0, 0); font-size: 18pt; font-weight: bold;">消息模型</span></font></div><div>消息模型其实早在应用在桌面应用程序中了。一个GUI程序的主线程就负责不停地读取消息并处理消息。所有的键盘、鼠标等消息都被发送到GUI程序的消息队列中，然后由GUI程序的主线程处理。</div><div><br/></div><div>由于GUI线程处理键盘、鼠标等消息的速度非常快，所以用户感觉不到延迟。某些时候，GUI线程在一个消息处理的过程中遇到问题导致一次消息处理时间过长，此时，用户会感觉到整个GUI程序停止响应了，敲键盘、点鼠标都没有反应。这种情况说明在消息模型中，处理一个消息必须非常迅速，否则，主线程将无法及时处理消息队列中的其他消息，导致程序看上去停止响应。</div><div><br/></div><div><br/></div><div><font style="font-size: 18pt;"><span style="color: rgb(173, 0, 0); font-size: 18pt; font-weight: bold;">如何解决同步IO必须等待IO操作</span></font></div><div>        当遇到IO操作时，代码只负责发出IO请求，不等待IO结果，然后直接结束本轮消息处理，进入下一轮消息处理过程。当IO操作完成后，将收到一条“IO完成”的消息，处理该消息时就可以直接获取IO操作结果。</div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><h1><span style="color: rgb(28, 51, 135);">异步IO（ asyncio） 协程</span></h1><div>   <a href="http://python.jobbole.com/87310/">http://python.jobbole.com/87310/</a></div><div><span>    <span>    </span></span> 无论多线程和多进程，IO的调度更多取决于系统，而协程的方式，调度来自用户，用户可以在函数中yield一个状态。使用协程可以实现高效的并发任务。</div><div>        实现协程的不仅仅是asyncio，tornado和gevent都实现了类似的功能。</div><div><br/></div><div><ul><li>event_loop 事件循环</li></ul></div><div style="box-sizing: border-box; padding: 8px; font-size: 12px; border-top-left-radius: 4px; border-top-right-radius: 4px; border-bottom-right-radius: 4px; border-bottom-left-radius: 4px; background-color: rgb(251, 250, 248); border: 1px solid rgba(0, 0, 0, 0.14902);"><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: 微软雅黑; font-size: 12pt;">程序开启一个无限的循环，程序员会把一些函数注册到事件循环上。</span></div><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: 微软雅黑; font-size: 12pt;">当满足事件发生的时候，调用相应的协程函数。</span></div></div><div><ul><li>coroutine 协程</li></ul></div><div style="box-sizing: border-box; padding: 8px; font-size: 12px; border-top-left-radius: 4px; border-top-right-radius: 4px; border-bottom-right-radius: 4px; border-bottom-left-radius: 4px; background-color: rgb(251, 250, 248); border: 1px solid rgba(0, 0, 0, 0.14902);"><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: 微软雅黑; font-size: 12pt;">协程对象，指一个使用async关键字定义的函数，它的调用不会立即执行函数，而是会返回一个协程对象。协程对象需要注册到事件循环，由事件循环调用。</span></div></div><div><ul><li>task 任务</li></ul><div style="box-sizing: border-box; padding: 8px; font-size: 12px; border-top-left-radius: 4px; border-top-right-radius: 4px; border-bottom-right-radius: 4px; border-bottom-left-radius: 4px; background-color: rgb(251, 250, 248); border: 1px solid rgba(0, 0, 0, 0.14902);"><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: 微软雅黑; font-size: 12pt;">一个协程对象就是一个原生可以挂起的函数，任务则是对协程进一步封装，其中包含任务的各种状态。</span></div></div></div><div><ul><li>future</li></ul><div style="box-sizing: border-box; padding: 8px; font-size: 12px; border-top-left-radius: 4px; border-top-right-radius: 4px; border-bottom-right-radius: 4px; border-bottom-left-radius: 4px; background-color: rgb(251, 250, 248); border: 1px solid rgba(0, 0, 0, 0.14902);"><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: 微软雅黑; font-size: 12pt;">代表将来执行或没有执行的任务的结果。它和task上没有本质的区别</span></div></div></div><div><ul><li>async/await 关键字</li></ul><div style="box-sizing: border-box; padding: 8px; font-size: 12px; border-top-left-radius: 4px; border-top-right-radius: 4px; border-bottom-right-radius: 4px; border-bottom-left-radius: 4px; background-color: rgb(251, 250, 248); border: 1px solid rgba(0, 0, 0, 0.14902);"><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: 微软雅黑; font-size: 12pt;">python3.5 用于定义协程的关键字，async定义一个协程，await用于挂起阻塞的异步调用接口。</span></div></div></div><div><br/></div><div><br/></div><h3><span style="color: rgb(28, 51, 135); font-size: 18pt;">定义一个协程</span></h3><div>使用async关键字，就像定义普通函数一样：</div><div style="box-sizing: border-box; padding: 8px; font-size: 12px; border-top-left-radius: 4px; border-top-right-radius: 4px; border-bottom-right-radius: 4px; border-bottom-left-radius: 4px; background-color: rgb(251, 250, 248); border: 1px solid rgba(0, 0, 0, 0.14902);"><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 12px;">import time</span></div><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 12px;">import asyncio</span></div><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 12px;">now = lambda : time.time()</span></div><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 12px;">async def do_some_work(x):</span></div><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 12px;">    print('Waiting: ', x)</span></div><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 12px;">start = now()</span></div><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 12px;">coroutine = do_some_work(2)</span></div><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 12px;">loop = asyncio.get_event_loop()</span></div><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 12px;">loop.run_until_complete(coroutine)</span></div><div><span style="background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 12px;">print('TIME: ', now() - start)</span></div></div><div><br/></div><div>        通过async关键字定义一个协程（coroutine），协程也是一种对象。协程不能直接运行，需要把协程加入到事件循环（loop），由后者在适当的时候调用协程。asyncio.get_event_loop方法可以创建一个事件循环，然后使用run_until_complete将协程注册到事件循环，并启动事件循环。因为本例只有一个协程，于是可以看见如下输出：</div><div style="box-sizing: border-box; padding: 8px; font-size: 12px; border-top-left-radius: 4px; border-top-right-radius: 4px; border-bottom-right-radius: 4px; border-bottom-left-radius: 4px; background-color: rgb(251, 250, 248); border: 1px solid rgba(0, 0, 0, 0.14902);"><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">Waiting:  2</span></div><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">TIME:  0.0004658699035644531</span></div></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><h3><span style="color: rgb(28, 51, 135); font-size: 18pt;">创建一个task</span></h3><div>        协程对象不能直接运行，在注册事件循环的时候，其实是run_until_complete方法将协程包装成为了一个任务（task）对象。所谓task对象是Future类的子类。保存了协程运行后的状态，用于未来获取协程的结果。</div><div style="box-sizing: border-box; padding: 8px; font-size: 12px; border-top-left-radius: 4px; border-top-right-radius: 4px; border-bottom-right-radius: 4px; border-bottom-left-radius: 4px; background-color: rgb(251, 250, 248); border: 1px solid rgba(0, 0, 0, 0.14902);"><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">import asyncio</span></div><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">import time</span></div><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;"> </span></div><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">now = lambda : time.time()</span></div><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;"> </span></div><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">async def do_some_work(x):</span></div><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">    print('Waiting: ', x)</span></div><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;"> </span></div><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">start = now()</span></div><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;"> </span></div><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">coroutine = do_some_work(2)</span></div><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">loop = asyncio.get_event_loop()</span></div><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;"># task = asyncio.ensure_future(coroutine)</span></div><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">task = loop.create_task(coroutine)</span></div><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">print(task)</span></div><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">loop.run_until_complete(task)</span></div><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">print(task)</span></div><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">print('TIME: ', now() - start)</span></div></div><div><br/></div><div>可以看到输出结果为：</div><div style="box-sizing: border-box; padding: 8px; font-size: 12px; border-top-left-radius: 4px; border-top-right-radius: 4px; border-bottom-right-radius: 4px; border-bottom-left-radius: 4px; background-color: rgb(251, 250, 248); border: 1px solid rgba(0, 0, 0, 0.14902);"><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">&lt;Task pending coro=&lt;do_some_work() running at /Users/ghost/Rsj217/python3.6/async/async-main.py:17&gt;&gt;</span></div><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">Waiting:  2</span></div><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">&lt;Task finished coro=&lt;do_some_work() done, defined at /Users/ghost/Rsj217/python3.6/async/async-main.py:17&gt; result=None&gt;</span></div><div style="font-size: 9pt;"><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">TIME:  0.0003490447998046875</span></div></div><div>        创建task后，task在加入事件循环之前是pending状态，因为do_some_work中没有耗时的阻塞操作，task很快就执行完毕了。后面打印的finished状态。</div><div>asyncio.ensure_future(coroutine) 和 loop.create_task(coroutine)都可以创建一个task，run_until_complete的参数是一个futrue对象。当传入一个协程，其内部会自动封装成task，task是Future的子类。isinstance(task, asyncio.Future)将会输出True。</div><div><br/></div><div><br/></div><div><br/></div><h3><font style="color: rgb(28, 51, 135); font-size: 18pt;">绑定回调</font></h3><div>绑定回调，在task执行完毕的时候可以获取执行的结果，回<font style="font-size: 12pt; color: rgb(173, 0, 0);"><b>调的最后一个参数是future对象，通过该对象可以获取协程返回值</b></font>。如果回调需要多个参数，可以通过偏函数导入。</div><div><br/></div><div style="box-sizing: border-box; padding: 8px; font-family: Monaco, Menlo, Consolas, &quot;Courier New&quot;, monospace; font-size: 12px; color: rgb(51, 51, 51); border-top-left-radius: 4px; border-top-right-radius: 4px; border-bottom-right-radius: 4px; border-bottom-left-radius: 4px; background-color: rgb(251, 250, 248); border: 1px solid rgba(0, 0, 0, 0.14902); background-position: initial initial; background-repeat: initial initial;-en-codeblock:true;"><div>import time</div><div>import asyncio</div><div>now = lambda : time.time()</div><div>async def do_some_work(x):</div><div>    print('Waiting: ', x)</div><div>    return 'Done after {}s'.format(x)</div><div>def callback(future):</div><div>    print('Callback: ', future.result())</div><div>start = now()</div><div>coroutine = do_some_work(2)</div><div>loop = asyncio.get_event_loop()</div><div>task = asyncio.ensure_future(coroutine)</div><div>task.add_done_callback(callback)</div><div>loop.run_until_complete(task)</div><div><span style="font-family: Monaco; font-size: 9pt; color: rgb(51, 51, 51);">print('TIME: ', now() - start)</span></div></div><div> <br/></div><div style="box-sizing: border-box; padding: 8px; font-family: Monaco, Menlo, Consolas, &quot;Courier New&quot;, monospace; font-size: 12px; color: rgb(51, 51, 51); border-top-left-radius: 4px; border-top-right-radius: 4px; border-bottom-right-radius: 4px; border-bottom-left-radius: 4px; background-color: rgb(251, 250, 248); border: 1px solid rgba(0, 0, 0, 0.14902); background-position: initial initial; background-repeat: initial initial;-en-codeblock:true;"><div style="font-family: Monaco; font-size: 9pt; color: rgb(51, 51, 51);"><span style="font-family: Monaco; font-size: 9pt; color: rgb(51, 51, 51);">def callback(t, future):</span></div><div style="font-family: Monaco; font-size: 9pt; color: rgb(51, 51, 51);"><span style="font-family: Monaco; font-size: 9pt; color: rgb(51, 51, 51);">    print('Callback:', t, future.result())</span></div><div style="font-family: Monaco; font-size: 9pt; color: rgb(51, 51, 51);"><span style="font-family: Monaco; font-size: 9pt; color: rgb(51, 51, 51);"> </span></div><div style="font-family: Monaco; font-size: 9pt; color: rgb(51, 51, 51);"><span style="font-family: Monaco; font-size: 9pt; color: rgb(51, 51, 51);">task.add_done_callback(functools.partial(callback, 2))</span></div></div><div><br/></div><div><br/></div><div><br/></div><h3><font style="font-size: 18pt; color: rgb(28, 51, 135);">future 与 result</font></h3><div><br/></div><div><br/></div><div><br/></div><h3><font style="font-size: 18pt; color: rgb(28, 51, 135);">阻塞和await</font></h3><div><br/></div><div><br/></div><h3><font style="font-size: 18pt; color: rgb(28, 51, 135);">并发和并行</font></h3><div><br/></div><div><br/></div><div><br/></div><div><br/></div><h3><font style="font-size: 18pt; color: rgb(28, 51, 135);">协程嵌套</font></h3><div><br/></div><div><br/></div><div><br/></div><h3><font style="font-size: 18pt; color: rgb(28, 51, 135);">协程停止</font></h3><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div></div><div><br/></div></span>
</div></body></html> 